Lecture 11 : Shannon vs . Hamming

In the last lecture, we proved the positive part of Shannon's capacity theorem for the BSC. We showed that by the probabilistic method, there exists an encoding function E and a decoding function D such that E m Pr noise e of BSCp [D(E(m) + e) = m] ≤ 2 −δ ′ n. (1) In other words, the average decoding error probability is small. However, we need to show that the maximum decoding error probability over all messages is small. In the last lecture, we quickly went over how (1) implies the latter. We will start today's lecture by going over this argument again. As was mentioned in the last lecture, the trick is to throw away all the messages that have high error probability. In particular, we only keep the messages with probability error at most the median error probability.